fix: implement exponential backoff to resolve EventStreamError #94
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
nboyse
added a commit
that referenced
this pull request
Feb 19, 2025
* add ne route * add new prompt * tidy up * add node to strip keyword * exponential backoff for summarisation * optimized prompt for summarisation * build(deps-dev): bump cryptography from 43.0.3 to 44.0.1 in /ai-tests (#82) Bumps [cryptography](https://github.com/pyca/cryptography) from 43.0.3 to 44.0.1. - [Changelog](https://github.com/pyca/cryptography/blob/main/CHANGELOG.rst) - [Commits](pyca/cryptography@43.0.3...44.0.1) --- updated-dependencies: - dependency-name: cryptography dependency-type: indirect ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * build(deps-dev): bump cryptography from 44.0.0 to 44.0.1 (#83) Bumps [cryptography](https://github.com/pyca/cryptography) from 44.0.0 to 44.0.1. - [Changelog](https://github.com/pyca/cryptography/blob/main/CHANGELOG.rst) - [Commits](pyca/cryptography@44.0.0...44.0.1) --- updated-dependencies: - dependency-name: cryptography dependency-type: indirect ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * use final_res to determine log * build(deps): bump cryptography from 43.0.3 to 44.0.1 in /django_app Bumps [cryptography](https://github.com/pyca/cryptography) from 43.0.3 to 44.0.1. - [Changelog](https://github.com/pyca/cryptography/blob/main/CHANGELOG.rst) - [Commits](pyca/cryptography@43.0.3...44.0.1) --- updated-dependencies: - dependency-name: cryptography dependency-type: indirect ... Signed-off-by: dependabot[bot] <[email protected]> * chore: branch naming convention (#86) * chore: branch naming convention * linting throughout codebase * update email/username relation * revert changes to retriever * rebase * linting * poetry has been messed with * build(deps): bump cryptography from 44.0.0 to 44.0.1 in /notebooks (#84) Bumps [cryptography](https://github.com/pyca/cryptography) from 44.0.0 to 44.0.1. - [Changelog](https://github.com/pyca/cryptography/blob/main/CHANGELOG.rst) - [Commits](pyca/cryptography@44.0.0...44.0.1) --- updated-dependencies: - dependency-name: cryptography dependency-type: indirect ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * fix: text_in_answer already exists and prevents further migrations * Revert "fix: text_in_answer already exists and prevents further migrations" * chore: fix redbox tests (#90) * fix: implement exponential backoff (#94) * Update homepage.html (#99) * Update homepage.html * the solution to the formatting issue --------- Co-authored-by: Natasha Boyse <[email protected]> * If documents are not selected, they should not be used (#101) * no documents returned if not selected * ruff --------- Co-authored-by: Saisakul Chernbumroong <[email protected]> * fix: prevent 504 for user table (#100) * fix: prevent 504 for user table * assertion is redundant * adding user id and chat id * formatting --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: Saisakul Chernbumroong <[email protected]> Co-authored-by: Saisakul Chernbumroong <[email protected]> Co-authored-by: nora-errouhly <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Nora Er-Rouhly <[email protected]> Co-authored-by: Saisakul Chernbumroong <[email protected]> Co-authored-by: zainabmohamoud <[email protected]>
larry6point6
pushed a commit
that referenced
this pull request
Feb 20, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Context
We are getting numerous flags from sentry about:
EventStreamError: An error occurred (serviceUnavailableException) when calling the InvokeModelWithResponseStreamWe dont want this anymore. Its probably because too many requests are being sent to bedrock, so exponential backoff is a healthy way to try mitigate this error.
Changes proposed in this pull request
Implement exponential backoff. Maybe further in future we should look at other errors to catch (ThrottlingException etc.)
Guidance to review
Check it is a suitable implementation
Relevant links
Things to check